rTop-k: A Statistical Estimation Approach to Distributed SGD

نویسندگان
چکیده

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Statistical inference using SGD

We present a novel method for frequentist statistical inference in M -estimation problems, based on stochastic gradient descent (SGD) with a fixed step size: we demonstrate that the average of such SGD sequences can be used for statistical inference, after proper scaling. An intuitive analysis using the OrnsteinUhlenbeck process suggests that such averages are asymptotically normal. From a prac...

متن کامل

A statistical approach to classify Skype traffic

Abstract- Skype is one of the most powerful and high-quality chat tools that allows its users to use of many services such as: transferring audio, sending messages, video conferencing and audio for free. Skype traffic has a lot of Internet traffic. Hence, Internet service providers need to identify traffic to do the quality of service and network management. On the other hand, Skype developers ...

متن کامل

Revisiting Distributed Synchronous SGD

Distributed training of deep learning models on large-scale training data is typically conducted with asynchronous stochastic optimization to maximize the rate of updates, at the cost of additional noise introduced from asynchrony. In contrast, the synchronous approach is often thought to be impractical due to idle time wasted on waiting for straggling workers. We revisit these conventional bel...

متن کامل

Accurate RTOP Estimation from PFG-NMR Signal

The return to the origin probability (RTOP) for diffusing molecules is a valuable indicator of porous media microstructure [1-2]. For example, in isolated pores with nonrelaxing walls, the pore volume is related to the RTOP at long diffusion times. Similarly, in disordered media, the temporal scaling of the RTOP is necessary in the estimation of the fractal dimension of the medium [3]. However,...

متن کامل

A Geometric Approach to Statistical Estimation

The role of Kerridge inaccuracy, Shannon entropy and Kullback-Leibler distance in statistical estimation is shown for both dicrete and continuous observations. The cases of data independence and regression-type dependence are considered in parallel. Pythagorean-like relations valid for probability distributions are presented and their importance for estimation under compressed data is indicated.

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: IEEE Journal on Selected Areas in Information Theory

سال: 2020

ISSN: 2641-8770

DOI: 10.1109/jsait.2020.3042094